Puppet shows have existed since antiquity in almost all countries of the world. In most forms, the puppet is manipulated either directly by hand, or by rods, strings or wires. Shadows of puppets are also sometimes used in a performance.
Perhaps the most advanced form of puppetry is Bunraku Puppetry. This form was originally developed at the end of the sixteenth century in Japan, and later took its name from Uemura Bunrakuken, a famous Japanese promoter in the early nineteenth century.
Bunraku puppetry requires teams of three to four puppeteers to stand behind each puppet and move the arms, legs, mouth and body using rods and the puppeteer's hands. The chief manipulator supports the puppet by inserting his or her left hand into the puppet body from behind, to move the puppet's head and mouth. His or her right hand controls the puppet's right hand. The assistants respectively take charge of the puppet's feet and left hand.
Since three or four people handle the different parts of the puppet, lifelike motion cannot be achieved without precision timing among the manipulators. For all motions, there are detailed rules and forms to be followed. No manipulator is allowed to act on their own.
In Japan, when operating this type of puppet on stage, the manipulators, as a rule, wear black gowns and black hoods. This signifies that the puppet is the main performer with the manipulator remaining behind the scenes. In the Japanese theatrical tradition, black costumes represent the invisible or nothingness.
It is one object of this invention to apply this form of advanced puppetry to the movie and video industry and make the manipulators truly invisible to the viewer in real time, while achieving lifelike movement of the puppets on the screen. This object is achieved through use of a significant improvement to what is known in the film industry as the “blue screen or key-color process.”
It is well known in the motion picture and video industry that two or more images can be combined into a single scene. This technique of combining images is primarily achieved through use of a key-color process in which one image is photographed against a solid key-color background, (such as blue, green or grey) and the second image is substituted in place of that key-color background. It is also known that the background can be various other colors, and that the substitution can be made electronically, or through optical photographic techniques. This process is also known as a keying process.
In such a known system, the image of an announcer, for example, is shot with a live camera in front of a blue screen as a foreground image. Subsequently, a background image is generated in real time as a result of graphics processing performed by a high speed computer, and is super-imposed on the foreground image. The super-imposed portion of the background image and the foreground image is blanked out by the keying process and a composite image is obtained by fitting an object's image portion of the foreground image into the blank portion.
Although there are many advantages to using the standard key-color process, there are also several disadvantages. For example, the key-color process is generally an iterative process in which the end product, or combined file or video is composited or “married in post” and is not viewable for evaluation until the entire process has been completed. As a result, film producers frequently view the end product and require changes to either the foreground or background portions, requiring that the entire sequence be “reshot.” Furthermore, because the process is post rendered in real-time, it is difficult to perform actual interaction between a moving puppet, for example, and the puppet's environment. The normal cycle for “blue screen” film production can require several iterations of this process. Various U.S. patents describe this process and variations thereof in greater detail, including U.S. Pat. Nos. 4,689,683, 5,479,597, 5,764,306, 5,949,433 and 6,034,740.
It is, therefore, a further object of this invention to provide a significant improvement to the known key-color process directed to the use of puppetry in the film and video industries utilizing virtual sets.
It is a still further object of this invention to provide an improved keying process which allows the use of an advanced form of puppetry with three or more puppeteers where the producer, the puppeteers, and the director can view the final product in real time as the live puppet show progresses.
It is another object of this invention to apply the inventive method and apparatus to film and video productions which do not necessarily include puppets and puppeteers.